"Coronavirus Antibody Tests: Can You Trust the Results?
A team of scientists worked around the clock to evaluate 14 antibody tests. A few worked as advertised. Most did not.
...For the past few weeks, more than 50 scientists have been working diligently to do something that the Food and Drug Administration mostly has not: Verifying that 14 coronavirus antibody tests now on the market actually deliver accurate results.
These tests are crucial to reopening the economy, but public health experts have raised urgent concerns about their quality. The new research, completed just days ago and posted online Friday, confirmed some of those fears: Of the 14 tests, only three delivered consistently reliable results. Even the best had some flaws.
The research has not been peer-reviewed and is subject to revision. But the results are already raising difficult questions about the course of the epidemic.
Surveys of residents in the Bay Area, Los Angeles and New York this week found that substantial percentages tested positive for antibodies to SARS-CoV-2, the official name of the new coronavirus. In New York City, the figure was said to be as high as 21 percent. Elsewhere, it was closer to 3 percent.
The idea that many residents in some parts of the country have already been exposed to the virus has wide implications. At the least, the finding could greatly complicate plans to reopen the economy.
Already Americans are scrambling to take antibody tests to see if they might escape lockdowns. Public health experts are wondering if those with positive results might be allowed to return to work.
But these tactics mean nothing if the test results can’t be trusted.
In the new research, researchers found that only one of the tests never delivered a so-called false positive — that is, it never mistakenly signaled antibodies in people who did not have them.
Two other tests did not deliver false-positive results 99 percent of the time.But the converse was not true. Even these three tests detected antibodies in infected people only 90 percent of the time, at best.
The false-positive metric is particularly important. The result may lead people to believe themselves immune to the virus when they are not, and to put themselves in danger by abandoning social distancing and other protective measures.
It is also the result on which scientists are most divided.
“There are multiple tests that look reasonable and promising,” said Dr. Alexander Marson, an immunologist at the University of California, San Francisco, and one of the project’s leaders. “That’s some reason for optimism.”
Dr. Marson is also an investigator in the Chan Zuckerberg Biohub, which partly funded the study. The results were published online on Friday; the research has not yet been peer-reviewed and may be revised.
Other scientists were less sanguine than Dr. Marson. Four of the tests produced false-positive rates ranging from 11 percent to 16 percent; many of the rest hovered around 5 percent.
“Those numbers are just unacceptable,” said Scott Hensley, a microbiologist at the University of Pennsylvania. “The tone of the paper is, ‘Look how good the tests are.’ But I look at these data, and I don’t really see that.”
The proportion of people in the United States who have been exposed to the coronavirus is likely to be 5 percent or less, Dr. Hensley said. “If your kit has a 3 percent false-positive, how do you interpret that? It’s basically impossible,” he said. “If your kit has 14 percent false positive, it’s useless.”
Dr. Hensley said the study nonetheless was well designed and the results pressing, given the sudden proliferation of antibody tests on the market and the push to use them to lift lockdowns.
“I think this is exactly the kind of study that we need right now,” he said.
Dr. Marson and his colleagues said they were drawn to the study for that very reason.
As universities in the Bay Area shut down all research not related to the coronavirus, some researchers began focusing on ways to improve diagnostic tests for SARS-CoV-2.
...A test made by Bioperfectus detected antibodies in 100 percent of the infected samples, but only after three weeks of infection. None of the tests did better than 80 percent until that time period, which was longer than expected, Dr. Hsu said.The lesson is that the tests are less likely to produce false negatives the longer ago the initial infection occurred, he said.
https://www.nytimes.com/2020/04/24/health/coronavirus-antibody-tests.html
A team of scientists worked around the clock to evaluate 14 antibody tests. A few worked as advertised. Most did not.
...For the past few weeks, more than 50 scientists have been working diligently to do something that the Food and Drug Administration mostly has not: Verifying that 14 coronavirus antibody tests now on the market actually deliver accurate results.
These tests are crucial to reopening the economy, but public health experts have raised urgent concerns about their quality. The new research, completed just days ago and posted online Friday, confirmed some of those fears: Of the 14 tests, only three delivered consistently reliable results. Even the best had some flaws.
The research has not been peer-reviewed and is subject to revision. But the results are already raising difficult questions about the course of the epidemic.
Surveys of residents in the Bay Area, Los Angeles and New York this week found that substantial percentages tested positive for antibodies to SARS-CoV-2, the official name of the new coronavirus. In New York City, the figure was said to be as high as 21 percent. Elsewhere, it was closer to 3 percent.
The idea that many residents in some parts of the country have already been exposed to the virus has wide implications. At the least, the finding could greatly complicate plans to reopen the economy.
Already Americans are scrambling to take antibody tests to see if they might escape lockdowns. Public health experts are wondering if those with positive results might be allowed to return to work.
But these tactics mean nothing if the test results can’t be trusted.
In the new research, researchers found that only one of the tests never delivered a so-called false positive — that is, it never mistakenly signaled antibodies in people who did not have them.
Two other tests did not deliver false-positive results 99 percent of the time.But the converse was not true. Even these three tests detected antibodies in infected people only 90 percent of the time, at best.
The false-positive metric is particularly important. The result may lead people to believe themselves immune to the virus when they are not, and to put themselves in danger by abandoning social distancing and other protective measures.
It is also the result on which scientists are most divided.
“There are multiple tests that look reasonable and promising,” said Dr. Alexander Marson, an immunologist at the University of California, San Francisco, and one of the project’s leaders. “That’s some reason for optimism.”
Dr. Marson is also an investigator in the Chan Zuckerberg Biohub, which partly funded the study. The results were published online on Friday; the research has not yet been peer-reviewed and may be revised.
Other scientists were less sanguine than Dr. Marson. Four of the tests produced false-positive rates ranging from 11 percent to 16 percent; many of the rest hovered around 5 percent.
“Those numbers are just unacceptable,” said Scott Hensley, a microbiologist at the University of Pennsylvania. “The tone of the paper is, ‘Look how good the tests are.’ But I look at these data, and I don’t really see that.”
The proportion of people in the United States who have been exposed to the coronavirus is likely to be 5 percent or less, Dr. Hensley said. “If your kit has a 3 percent false-positive, how do you interpret that? It’s basically impossible,” he said. “If your kit has 14 percent false positive, it’s useless.”
Dr. Hensley said the study nonetheless was well designed and the results pressing, given the sudden proliferation of antibody tests on the market and the push to use them to lift lockdowns.
“I think this is exactly the kind of study that we need right now,” he said.
Dr. Marson and his colleagues said they were drawn to the study for that very reason.
As universities in the Bay Area shut down all research not related to the coronavirus, some researchers began focusing on ways to improve diagnostic tests for SARS-CoV-2.
...A test made by Bioperfectus detected antibodies in 100 percent of the infected samples, but only after three weeks of infection. None of the tests did better than 80 percent until that time period, which was longer than expected, Dr. Hsu said.The lesson is that the tests are less likely to produce false negatives the longer ago the initial infection occurred, he said.
https://www.nytimes.com/2020/04/24/health/coronavirus-antibody-tests.html
